Making Sense of Implicit User Activity within Social Web Video with Stochastic Pulse Modeling

In this paper, we present a method that detects patterns of user activity on a web video, in order to extract video scenes. Previous research in video retrieval has focused on content-based techniques, such as pattern recognition algorithms that attempt to understand the low-level features of a video. There have also been user-based approaches, but they have assumed an extra effort by the users, such as comments, tags, and annotations. We are proposing a stochastic pulse modeling method, which makes sense of a web video by analyzing implicit user interactions, such as pause and thirty seconds rewind. In particular, we have modeled the user information seeking behavior as a continuous signal and the semantic regions as a discrete pulse of fixed width. Then, we have devised a controlled user experiment with two types of information-rich videos (lecture, documentary) and calculated the correlation coefficient between dynamically detected pulses at the local maximums of the user activity signal and the pulse of reference. We have found that if the users are actively seeking for information in a video, then their activity significantly matches the semantics of the video. This research complements previous work on content-based information retrieval and provides an additional user-based dimension for modeling the semantics of a social web video.